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Abstract 

We prove existence and uniqueness of the minimizer for the average 
geodesic distance to the points of a geodesically convex set on the sphere. 
This implies a corresponding existence and uniqueness result for an opti- 
mal algorithm for halfspace learning, when data and target functions are 
drawn from the uniform distribution. 



1 Introduction 

Let iS"~^ be the unit sphere in M" with normahzed uniform measure tr and 
geodesic metric p, and let K he a proper convex cone with nonempty interior 
in M". We will show that the function ^ : R defined by 



'^kM = / p{w,y)da{y) 

attains its global minimum at a unique point on 5"^^. While existence of the 
minimum is straightforward, uniqueness seems surprisingly difficult to prove. 

A similar problem has been considered in [5] and [T]. In these works the 
intention is to define a centroid, so integration is replaced by finite summation 
and p {w, y) replaced by p {w, y)^ . Since the problem is rather obvious, it appears 
likely that a proof of the above result exists somewhere in the literature and we 
just haven't been able to find it. 



2 Optimal halfspace learning 

Our motivation to consider this problem arises in learning theory. Specifically 
we consider an experiment, where 

1. A unit vector u is drawn at random from a and kept concealed from the 
learner. 
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2. A sample x = {xi,...,Xm) € («S" ^)"* is generated in m independent 

random trials of cr. 

3. A label vector y = m (x) S { — 1, 1}™ is generated according to the rule yi = 
sign{{u,Xi)), where (., .) is the euclidean inner product and sign{t) = 1 
if t > and — 1 if i < 0. The sign of is irrelevant, because it corresponds 

to events of probability zero. 

4. The labeled sample (x,y) — (x, ?i(x)) is supplied to the learner. 

5. The learner produces a hypothesis /(x, y) G according to some 
learning rule / : x {-1, 1}™ ^ 

6. An unlabeled test point x e >S"~^ is drawn at random from a and pre- 
sented to the learner who produces the label y = sign ((/ (x, y) , a;)). 

7. If sign {{u,Xi)) = y the learner is rewarded one unit, otherwise a penalty 
of one unit is incurred. 

We now ask the following question: Which learning rule / will give the 
highest average reward on a very large number of independent repetitions of 
this experiment? 

Evidently the optimal learning rule has to minimize the following functional: 
^ (/) = Eu^ctEx-ct'" Pr {sign ((/ (x, u (x)) , x)) ^ sign {{u, x))} . 

Now a simple geometric argument shows that for any v,u € <S"~^ we have 
Pr {sign{{v,x)) 7^ sign{{u,x))} = p{v,u) /w, 

relating the misclassification probability to the geodesic distance. For a labeled 
sample (x,y) e (<S"-i)" x {-1, 1}" we denote 

C7(x,y) = {«e5"-i:n(x)=y}. 

C (x, y) is thus the set of all hypotheses consistent with the labeled sample 
(x, y). Observe that, given x and u there is exactly one y such that y = u (x), 
that is u e C (x, y). We also have C (x, y) = K (x, y) n <S"~^ where K (x, y) is 
the closed convex cone 

K{x,y) = {veW : {u,yiXi) > 0,V1 < i < m} . 

We therefore obtain 

f2(/) = 7r-ilE„^^Ex^^n.p(/(x,n(x)),u) 

= TT-^Ex^a- 1E„^<7P(/(X,«(X)) ,U) lc(x,y) («) 

yGi-l,!}" 

= 7r"^Ex-<7-« ^ E„^(^p (/ (x, y) , w) lc(x,y) (w) 
yef-i,!}"" 

= TT-^Ex^a- X) V'K(x,y) (/(x,y))• 
ye{-l,lr 
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If K (x, y) has empty interior then the corresponding summand vanishes, so we 
can assume that K (x, y) has nonempty interior. Clearly —yiXi ^ K (x, y) for 
all example points, so K (x, y) is a proper cone. Our result therefore applies 
and asserts the existence of a unique minimizer /* (x, y) of the function V'ii'(x y) ■ 
The map /* : (x, y) /* (x, y) is then the unique optimal learning algorithm. 

The map /* also has the symmetry property /* (l^x, y) = V f* (x, y) for any 
unitary V on M". This is so, because 

as is easily verified. We will also show, that the solution /* (x, y) must lie in 
the cone 

and that ipxi^ y) i^'^ other local minima. 

3 Proof of the main result 

Notation 1 .) is the geodesic distance and a the Haar measure on 5"^^. 
For A C R" we denote Ai ^ {x e A : \\x\\ = 1} = AnS''^'^. 'Cone' will always 
mean 'convex cone '. For A C K." we denote 

A^{x : {x,v) > 0,Vt; e A} . 

This is always a closed convex set. A proper cone K is one which is contained 
in some closed half space. For a set A the indicator function of A will be denoted 
by Ia 



Lemma 2 Let K be a closed cone 

(i) If w ^ K then there is a unit vector z G M" such that {z,w) < and 
{z, y) >0 for all y e K. 

(ii) [fdj = K . 

(Hi) Suppose that K is proper and has nonempty interior, w G 5"^^ , w ^ 
KU (^^^^ 'J^'^ e > 0. Then there exists z with \\z\\ = 1 such that —e < {z, w) < 
and {z, y) > for all y E K\ {0}. 

Proof, (i) Let B be an open ball containing w such that K Ci B = (d. Define 

O = {Xx : X E B,X> 0} . 

Then K and O are nonempty disjoint convex sets and O is open. By the Hahn- 
Banach theorem (11, Theorem 3.4) there is 7 G K. and z G R" such that 

(z, x) < 7 < (z.y) ,\/x E 0,y E K. 
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Choosing y = G K gives 7 < 0, letting A ^ in {z, Xw) < 7 shows 7 > 0, so 
that 7 = 0. The normalization is trivial. 



(ii) Trivially K C [Kj ■ On the other hand, if w ^ K let z be the vector 

from part (i). Then z £ K but {w, z) < 0, so that w ^ (^^^ ■ 

(iii) Since w ^ K there exists xi K s.t. {w,xi) < 0. Since the interior of 
K is nonempty, K is the closure of its interior (Theorem 6.3 in [3]), so we can 

assume xi G mt{K). Similarily, since w ^ (^—K^ we have —w ^ K, so there is 

X2 G mt{K) with {—w,X2) < 0, that is {w,X2) > 0. Since the interior of K is 
convex it contains the segment [xi, X2], so by continuity of (w, •) there is some 
a;o G int{K) with {w,xo) = 0. Since isT is a proper cone mt{K) and we can 
assume that ||xo|| = 1. 

Let c > be such that x' £ K whenever \\xq — x'\\ < c. We define 

z = (1 — 77) xq ~ rj'^^^w, where < 77 < min < -j-^ — 

Since {w,Xo) = it is clear that z is a unit vector. Also (w, z) = — ry^/^ > — e, 
and for any y e i^i we have Xq — cy E K, so (y, Xq — cy) > and 

{y,z) = (1 -T])^^^ {{y,XQ - cy) +c{y,y)) -T]^/^ {y,w) 

> (l-??)l/'c-7?l/2 >0. 



Theorem 3 Let K C M""i be a closed proper cone with nonempty interior, 
g : [0,tt] —>■ R continuous and the function ip : 5""^ —>■ M defined by 



tp{w) = / g{p{w,y))da{y). 

JKi 

(i) ip attains its global minimum on S"^^. 

(ii) If g is increasing then every local minimum of ip must lie in K U (^^K^ 

and every global minimum of must lie in K Cl K . 

(iii) If g is increasing and convex in [0, 7r/2] then the global minimum of ^ 
is unique and corresponds to the only local minimum outside —K. 

(iv) If g is increasing, convex in [0, 7r/2] and concave in [7r/2,7r] then the 
global minimum of ip is unique and corresponds to its only local minimum on 

Proof, (i) is immediate since 5"^^ is compact and is continuous. 



(ii) Fix w e 5"-i, wf^ku (-i') . 



We will first show that there can be no 
local minimum of ip at w. Let e > be arbitrary and choose z as in the lemma 
(iii). The functional z divides the sphere S"^^ into two open hemispheres 

L = {u: (z, u) < 0} and R ^ {u : (z, li) > 0} , 
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and an equator of cr-measure zero. Note that w £ L and Ki C R. We can write 

c = min {y, z) > 0, 
yeiii 

since Ki is compact and y i-^ {y, z) is continuous. With V we denote the 
reflection operator which exchanges points of L and R 

Vx — — {x, z) z + (x — (x, z) z) . 

V is easily verified to an isometry and V"^ = I. 

Suppose now that u £ R and Vu 6 K. We claim that u is in the interior of 
K. Indeed, if u' G M" satisfies ||m — < 2 {u, z) c, then for all y G Ki we have 

{u',y) = {u,y) - {u-u',y) > {u,y) -2{u,z)c 
> {u,y)~2{u,z){z,y) = {Vu,y)>Q, 

so u' G {/^^ ~ P*'^^ lemma. This establishes the claim and 

shows that V (K) n i? is contained in the interior of K. It follows that 

Vu G R, Ik (u) > Ik (Vu) . (1) 

Also V (K) n i? is relatively closed in R while int{K) n i? is open in R. Since R 
is connected they can only coincide if V (K) Ci R = R. But this is impossible, 
since then 

LUR = V{V{K)nR)U{V{K)nR)CV{V{Kr\L))Umt{K) 
= {KnL)U int {K) C K, 

and K is assumed to be a proper cone. So V {K) n i? is a proper subset of 
int(iir) n R. The inequality ^ is therefore strict on the nonempty open set 
{mt{K)nR)\{V {K)nR). 

Using isometry and unipotence of V we now obtain 

tp (w) ~ tp (Vw) = / {g{p{w,u)) ~ g {p{Vw,u)))1k {u)da (u) + 

JR. 

+ J (dip {w, u)) - g{p (Vw, u))) Ik (u) da (u) 
= [ igip{w,u))~g{p{Vw,u))){lKiu)-lKiVu))daiu) 

JR 
> 0. 

The inequality holds, because the first factor [g {p {w, u)) ~ g {p (Vw, u))) in the 
last integral is always positive for u € R, since g is increasing and p is increasing 
in the euclidean distance. The second is nonnegative and positive on a set of 
positive measure. Since \\w — Vw\\ = 2e and e > was arbitrary, we see that 
every neighborhood of w contains a point where ip has a smaller value than at 
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w. Wc conclude that w cannot be a local minimum of ^jj, which proves the first 
assertion of (ii). 

li w ^ K choose z as in part (i) of the lemma and let W be the isom- 

etry Wx = —{x,z)z + {x — {x,z)z). The Vm G X wc have g{p{w,u)) > 
g {p [Ww, u)), so ip (w) > 2p (Ww) and w cannot be a global minimizer of ip. So 

every global minimizer must be in if n (^K C] ^—K^^. Since Ki n ^—Ki^ is 

obviously empty the second assertion of (ii) follows. 

(iii) Now let wi,W2 G Ki with ?i>i ^ W2. Connect them with a geodesic in 
Ki and let w* G Ki be the midpoint of this geodesic, such that p {wi ,w*) = 
p {w*,'W2) = p (wi, W2) /2 < tt/2. We define a map U by 

Ux = {x, w*) w* — {x — {x, w*)w*) . 

Geometrically U is reflection on the one-dimensional subspace generated by w* . 
Note that W2 = Uwi and that p {u, Uu) — 2p {u, w*) if p {u, w*) < 7r/2 and that 
p{u,Uu) = 2tt - 2p{u,w*) if p{u,w*) > it/2. 

Take any u € K^. Since w* G Ki we have p (u, w*) < 27r, whence p {u,Uu) = 
2p {u, w*). All the four points wi, W2, u and Uu have at most distance 7r/2 from 
w* and lie therefore together with w* on a common hemisphere. By the triangle 
inequality 

2p{u,w*) = p{u,Uu) 

< p {u, wi) + p {wi, Uu) = p {u, wi) + p {Uwi,UUu) 

= p{u,Wi) + p{W2,u) . 

If u does not lie on the geodesic through wi and W2 and not at distance 7r/2 from 
w* strict inequality holds, and since Ki has nonempty interior strict inequality 
holds on an open subset of Ki. If p is increasing and convex in [0,7r/2] then 
dividing by 2, applying g and integrating over Ki we get 

^{w*) < (1/2) (V- iwi)+ij{W2)). 

It follows that there can be at most one point in Ki where the gradient of V' 

vanishes, and this point, if it exists, must correspond to a local minimum. By 
(ii) this is the unique global minimum and the only local minimum outside —K, 
which establishes (iii). 

(iv) If xi,X2 G —Ki and x* G ^Ki is their midpoint, then for u G K we 
obtain, using p [xi, u) ^ tt ~ p {—Xi, u) and a reasoning analogous to the above, 

p (u, w*) > (1/2) {p (u, wi) + p (u, ii;2)) , 

the inequality being again strict on a set of positive measure and preserved 
under application of a function g which is increasing and concave in [7r/2,7r], so 
that 

^{w*) > (1/2) ii, iwi)+iP{w2)). 

It again follows that there can be at most one point in —Ki where the gradient 
of tjj vanishes, and this point must now correspond to a local maximum. We 
conclude that ip has a unique local minimum which lies in /sTi. ■ 
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Remark. An example of a function as in (iii) is g (t) =t^,m. which case the 
minimizcr is the spherical mass centroid considered in [2] and [1]. Examples of 
functions as in (iv) are of course the identity function, in which case we obtain 
the result stated in the introduction. We could also set g{t) = 2 (1 — cost), in 
which case the function reads 

V'(w) = / \\w - y\\^ da {y) . 

JKi 

In this case uniqueness of the minimum can be established with much simpler 
methods. 
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